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lackgrnnnd qf Invent^ 


Using voice recognition to command devices is a well known art but it suffers from a 
deteriorating recognition rates when it comes to recognition of speaker independent 
words since it is hard to coma up with a recognizer chat may recognize words spoken by 
different people in adverse environments*. The following invention offers a solution to 
this problem by flexing combined handwriting and speaker dependent recognition in 
order to achieve high recognition credibility essential for proper work in a vehicle 
environment 


Description of invention 

The invention . referred to from now on a* "Hmartcar" , is a device that enables the 
operation of systems and auxiliary devices ^intended for use in transportation vehicles, 
by using voice or/and handwritten commands. It combines voice recognition and 
handwriting recognition technology thus providing a reliable and robust method of 
interacting with those systems or devices^ >. 

For example in a car the smartcar may operate .by using vo ice or /and handwritten 
commands only, devices such as cellular;phone (caikit), navigation system, a PC, radio, 
arraro system , electrical windows ,etc 

More specific uses may be voice dialing (using a name), number dialing (saying the 
digits or writing them), getting E-mails, operating (activating and deactivating) the alarm 
system using a written password, and many more, (refer to figure 1) 

Principle of operation 

Input commands (figure 2) . ^ 

i«: ; 

As can be seen in figure 2, The smartcar Receives voice commands through a microphone. 
The microphone receives the human voice add transforms it to an electrical signal. Hiis 
signal is then amplified and digitised. After jdigitization the voice recognizer utilizes 
processing software in order to process the^ommand and perform recognition by 
comparing the command to previous pre-traincd commands stored in a library which 
resides on a memory chip , ' 

The voice recognition is done by the voice recognizer module and after it recognizes the 
command it sends it to the main program software module. 

Handwritten commands are received through a touchpad connected to the smartcar. 
The touchpad is a device that whenever touched at a certain point it transmits the point' s 
X and Y coordinates as a string of digital signals. A touchpad may refer to any touch 
sensitive device (e.g. touch screen)- la thfeceac of the touchpad the input may be done 
with the finger. The handwriting recognizer ^receives the input character or gesture or 
word written on the touchpad and compares it to pre-traincd characters library. Once the 

**'-«| 


recognitor found b match it notifies the main program module as it is with the voice 
recognize. ) 

The library may come with pretrained symbols (alphanumeric characters or any other 
gestures) already installed so the user does not have to train the recognizer to rec gnize 
them* Each f the symbols (either pre-traine^ or not), can be trained by the user to fit his 
or her own style. 

Training iv- 

Training both recognizers means associating**?, spoken word or a written character to a 
command that is understood by one of the devices for example the gesture "arrow up** 
would mean closing the windows, or spcakiiig one's name would activate the cellular 
phone to dial that person's number. It is essential to know the communication protocol of 
the devices attached to the smartcar in order tb associate the command with an operation 
or, in case of operating the vehicle's systems 'such as an alarm or a window , it is essential 
to know the actual signal that operates these system*. 

This information may be supplied by the devices manufacturers and by the cars 
manufacturers l ^4' 

Operation (figures 23) '->j ' 

* *c 

The global operation of the smartcar is controlled by a switch or push button. 
The smartcar has two major operation mode), which are recognition and training modes. 
Switching between the modes is done by thi switch. For example if the user warns the 
smartcar to enter recognition mode he wiUi^iok the switch and for now on the smartcar 
will accept commands ~- spoken or written:^i>es. If the user wants to tain the smartcar 
for a new command he may double-clickthe'fwitch so the main program may switch to 
training mode. 4r 

Other global commands may be associatedftyith the switch and which are pre-dctermined 
such as ending a conversation conducted over the cellular phone or closing an application 
window on the computer. 

The function of the main program is to combine and coordinate the recognition modules 
in order to produce the proper command to die chosen device. The first command after 
entering the recognition mode should be die name of the device we want to communicate 
with. After thai the main program will switch to the chosen device sending commands 
recognized either by the voice recognizer :ci, the handwriting recognizer to that specific 
device. After the device is chosen the matogrograxn sends commands recognized by both 
recognizers to that device. It is the role of thjb main program to associate the proper 
device command with the recognized command given the state of the communication 
between the gmt^t^w and the specific deviDtv.The user may receive a feedback from the 
smartcar indicating the recognized commaa&through a loudspeaker attached to it. The 
same gesture or spoken command may haye-i different meaning interpreted by the main 
program depending on the device the smartcar is connected to. For example the spoken 
word "up" may mean M close window 1 if the car's system is connected but it can also 
mean move to radio station up scale. s.;r,i 


The user may define, or use predefined amps (wu of commands) that may refer to 
more than one device attached to the ssiarcat. These macros may be operated regardless 
of the device that the smattcar is cuavjfttlj^Uneted to. 

For example, the us r may say the c mmajiarcios© car" and as a result, the smartoar will 
olose the windows, shut the radio off. shut jg^ cellular phone oft lock the doora and 
activate the alarm system. Ail these may o^Ur, for example, while Che system expects for 
a radio channel voice command, or a dig^tplbe written. 

The interface module, implemented in hardware and software, is the physical module all 
the devices connect to through electrical ^oimectoxa. It is controlled by the main program 
and it selects the actual lines mat a command will be sent upon to the device. This 
module also produces the correct voltage arid! current needed to communicate with the 
devices. For example if the user communicates with a PC via RS232 protocol then 
translation of digital commands pro duced|by^tbo main program into actual +M2 volts 
pulses (as required by the protocol) will hc^^rformed by me interface unit. 

For example: J^h 

After a click the user may say the word "M*. The main program will now route all the 
recognized commands to the radio . The u$mmay write or say (if so trained) the number 
u 5" The main program will translate the re^gnized u 5" to a digital command understood 
by the radio as "jump to station number ^('instead of pressing the "5" on the radio 
itself) and send it through the interface to\w radio. 

In case of connecting to an alarm system ^^he vehicle, the user may train double 
passwords: voiced and handwritten as carhfee seen in figure 3, And only by recognizing 
both passwords the main program may sends* command to a switch activating or 
deactivating the alarm system. The cctrnrrirtn^l may be in a term of anelectrical pulse 
activating the switch 

In ease of operating a car navigation systam W user may activate the proper menu of that 
system by voice and then enter names of ; t^|ts by writing it on the touchpad. 

The software may run on a CPU and the Caries may reside on memory chips such as 
flash memory or Eproms. 

Power ujfy 



The smartcar may operate on the car's v^ga^e . connecting it to the lighter outlet, or on 
batteries. 


Reference* 

The v ice recognizer may bs utilized by ARTb - (Advanced Recognition Technology 
Inc.) voice recognizer as described in U.S^jatcnt application N . 0862565 1 . Or by any 
other voice recognition module. nj* 

The handwriting recognizer may be utilufed'by ARTs handwriting recogni26r as 
described In patent applications 0*528923, 08418530, 08282187, 08428806, and U.S. 
patent No. 5,559,633. 

Reference Ib also made to ART'S U.S. patten application 08878741. 

* .* .> 

Combination of handwriting and voice recognition annihilates the need for speaker 
independent voice recognition technology »iwhich today is hard to implement in a noisy 
environment 

Handwriting is not influenced by the car*£ wisy environment. 

You can command and operate in privacy ^wjtile other persons create noise. 

Hie user may activate devices which need;kjptfx voice and handwriting commands such as 
personal computers or cellular phones (sending short text messages). 

The user may operate the devices safely >Vh^e driving. 

Mstpfpgoaa 

Figure 1: smartcar general description 
Figure 2: amartcar block diagram 

Figure 3: Alarm system application ^ r.i 

Figure 4: Principal of operation .J ; . . 






